Abstract Title Page 

Not included in page count. 



Title: Deconstructing the Instruction of the Control of Variables Strategy: Key Components of 
Science Instruction 

Author(s): Benjamin D. Freer 1 , Elizabeth P. Lorch 1 , Robert F. Lorch, Jr. 1 , & William 
Calderhead 2 



Affiliation of all authors: 1 Children at Risk Research Cluster, Department of Psychology, 
University of Kentucky 

2 Department of Special Education and Rehabilitation Counseling, 
University of Kentucky 

Email addresses for all authors: in order: benjamindfreer@gmail.com 

elorch @ email . uky . edu 
rlorch @ email . uky .edu 
William.Calderhead@uky.edu 

Contact email for paper: Benjamin Freer, benjamindfreer@gmail.com 



SREE Fall 2011 Conference Abstract Template 




Abstract Body 

Limit 4 pages single spaced. 



Background / Context: 

Description of prior research and its intellectual context. 



A crucial element of the logic of the scientific method is the Control of Variables 
Strategy (CVS), in which conclusions about causal relations are enabled by manipulating a focal 
variable of interest while holding all other variables constant. Understanding CVS in elementary 
grades generally is poor, but can be improved with systematic instruction. Chen and Klahr 
(1999) developed a brief teaching intervention combining direct instruction in CVS with hands- 
on experience in constructing experiments. The method has been shown to be effective in 
individual instruction and in classroom settings (Klahr & Nigam, 2004; Lorch et al., 2010). 
Although relatively brief, the intervention is sufficiently complex that questions remain 
concerning which of its components are critical to success. Further, the intervention is effective 
in classrooms that are relatively high-achieving in science, but less effective in lower-achieving 
classrooms (Lorch et al., 2010). In our program of research, we have “deconstructed” the 
teaching intervention to determine its most important components and to examine strategies for 
improving learning for all students. 

Purpose / Objective / Research Question / Focus of Study: 

Description of the focus of the research. 



The studies to be discussed in this presentation center on two questions: What 
components of the Chen and Klahr (1999) teaching intervention are most important in producing 
and maintaining children’s understanding of CVS? How can we build from this knowledge to 
improve understanding of CVS for all students? 

Setting: 

Description of the research location. 



The studies discussed in the symposium all were conducted in public elementary schools. 
Some examined classroom-based interventions that were delivered as part of regular instruction 
in 4 th grade classrooms of 20-28 students. Other studies involved individual sessions with 3 rd or 
4 th grade students, either during the school day or as part of an after-school program. 

Population / Participants / Subjects: 

Description of the participants in the study: who , how many, key features, or characteristics. 



Classroom-based studies were conducted with 4 th grade children from 96 classrooms in 
public elementary schools. Schools were selected to represent high average (schools that 
achieved highest scores in the district) or low average (schools that achieved the lowest scores in 
the district) scores on a statewide standardized test of science achievement. Studies involving 
individual sessions sampled 3 ld or 4 th grade students (who had not participated in classroom- 
based studies) from these and other local elementary schools. Across studies discussed in this 
presentation, nearly 3000 children participated. 
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Intervention / Program / Practice: 

Description of the intervention, program, or practice, including details of administration and duration. 
For Track 2, this may include the development and validation of a measurement instrument. 



Because multiple studies are discussed, the “typical” version of the intervention is 
described here, with variations relevant to specific research questions explained in the Research 
Design section. For all students, instruction centered on an example experimental situation, in 
which students could test whether a specific variable influenced how far a ball rolled from an 
inclined plane (ramp). The variables that could be manipulated included the (1) Steepness of the 
ramp (high or low); (2) Surface of the ramp (rough or smooth); (3) Length of the ball’s run (long 
or short, determined by the starting point); Type of ball (in some studies new or used; in other 
studies large or small). On Day 1 of a classroom-based study or in the first part of an individual 
session, students completed a pretest of the ability to identify valid experiments (binary-choice 
test), then were introduced to the ramps variables and completed a pretest of the ability to design 
valid experiments (ramps design test). On Day 2 of a classroom-based study or in the second 
part of an individual session, students received CVS instruction appropriate to the experimental 
conditions of a given study. In the typical version of the instructional intervention, children first 
are shown examples of confounded designs. Instruction focuses on getting children to identify 
problems with making conclusions based on a confounded design, then led through correction of 
the design to create an unconfounded experiment. After working through a couple of these 
examples, the instructor summarizes a general principle for CVS. On Day 3 of a classroom- 
based study or in the last part of an individual session, children complete posttests of CVS 
understanding. In a subset of studies, children are tested again after a delay of 4 months to 2 l A 
years. 

Research Design: 

Description of the research design. 



The four studies to be presented examined the effects of specific components of CVS 
instruction, whether specific components varied in effects for students from schools with high 
average science achievement scores and students from schools with low average science 
achievement scores, and how components of instruction influenced maintenance of learning after 
delays of months or years. Study 1 was a classroom-based experiment that examined the 
separate contributions of guided instruction and hands-on experimentation to learning CVS and 
whether these effects differ by school science achievement level. The intervention was modified 
to create three instructional conditions: 1) CVS instruction with no experience designing ramps, 
2) no CVS instruction but students were given experience designing ramps experiments, and 3) 
CVS instruction and students were given experience designing ramps experiments. Study 2 
modified the third condition of Study 1 to examine, in an individual setting, the degree to which 
the instructor’s elicitation of student responses influenced learning, as compared to a condition in 
which the instructor delivered the same information without asking for student responses. Study 
3 was a large-scale classroom-based study that examined the effect of the type of examples used 
by the teacher during CVS instruction; the order in which instruction, ramps design experience, 
and testing were provided; whether the effects of the type of instructional examples differed by 
school science achievement level; and whether learning was maintained during delays of several 
months or 2.5 years. In the “negative” examples condition, instruction began as described earlier, 
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by presenting fully confounded experiments that the instructor and child corrected (“negative” 
examples). In the “positive” examples condition, the instructor taught CVS by presenting only 
unconfounded experiments and showing how these designs captured the principle of the CVS. 

In the “control” condition, children designed ramps experiments and completed binary-choice 
tests (posttests for the other conditions) before receiving CVS instruction, half with “negative” 
and half with “positive” examples, on Day 3. Delayed testing occurred after 4-5 months and 
after 2.5 years. Finally, Study 4 investigated whether the number of variables demonstrated 
during one-on-one CVS instruction (4 variables vs. 2 variables) or the opportunity to plan 
designs in advance would influence learning of children from schools scoring low in average 
science achievement. Children who planned designs in advance were provided with a packet that 
could function to organize the design prior to building their ramps experiments. The children 
decided how to design their ramps on paper first and then built their ramps to match their design 
as they had recorded in the packet. 

Data Collection and Analysis: 

Description o f the methods for collecting and analyzing data. 

For Track 2, this may include the use of existing datasets. 



As noted above, children generally completed two forms of pretests and two forms of 
posttests. One form of testing asked students to discriminate valid from invalid experiments 
(binary-choice test) and was scored in terms of the percentage of correct answers and whether 
the child demonstrated mastery of CVS (above 85% correct). The other tested the ability to 
design valid experiments (ramps design test) and was scored in terms of the percentage of test 
designs that were entirely correct (appropriate pairings on all 4 variables). Children who 
demonstrated mastery of CVS during pretesting were excluded from analyses. Children retained 
in the analyses averaged chance level performance on the pretests (e.g., 50% correct on binary- 
choice tests). Data obtained from classroom-based studies were analyzed with HLM to account 
for the nested structure of students within classrooms within schools. Data obtained from 
individual sessions were analyzed with analyses of variance. 

Findings / Results: 

Description of the main findings with specific details. 



For simplicity, the findings presented for each study are in terms of percentage of correct 
answers on the binary-choice test of CVS understanding. Consistent findings emerge for the 
other measures noted above. 

Study 1: Conditions defined by the presence of guided instruction and/or the 
opportunity to design ramps experiments. Learning across the three instructional conditions 
was ordered: guided instruction with opportunity to design ramps (72% correct) > guided 
instruction only (65%) > opportunity to design ramps only (57%). Although children from 
schools with high average science achievement scores demonstrated more learning than children 
from low average science achievement scores (e.g., 82% correct (high) and 63% correct (low) 
for guided instruction with opportunity to design ramps), the pattern of results across conditions 
was the same for both school environments. Differences in CVS understanding across 
conditions remained several months later, with little change in performance after delay. 

Study 2: Role of interaction during instruction. Learning was greater in the 
instructional condition that included interaction (80% correct) than in the one where the same 
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information was delivered by the instructor without eliciting student interaction, even though the 
individual situation ensured student attention in both conditions (70% correct). 

Study 3: Instruction with “negative” examples corrected during the lesson vs. 
instruction centering on “positive” unconfounded designs only; instruction before or after 
work with ramps and testing. As in Study 1, children from schools with higher average science 
achievement scores learned more than those from lower average science achievement scores. 
More importantly, the pattern of results differed for the two school environments. Children from 
schools with higher average science achievement scores demonstrated greater immediate (Day 3) 
learning (84% correct) when presented with “negative” (confounded) examples and guided to 
correct the designs than when presented with “positive” (unconfounded) examples only (73% 
correct). However, differences between the conditions were not present after several months or 
after 2-1/2 years. Children from schools with lower average science achievement scores also 
showed better learning from the “negative” examples condition than from the “positive 
examples” condition at immediate testing (69% correct compared to 56% correct) and this 
difference remained after several months and after 2-1/2 years. Providing CVS instruction after 
the opportunity to design ramps and complete initial testing (at the end of Day 3 for the “control” 
condition) resulted in less learning than the “negative examples” and “positive examples only” 
conditions in schools with higher average science scores and less learning than the “negative 
examples” condition in schools with lower average science scores. 

Study 4: Effect of number of variables used in teaching and the opportunity to plan 
ramps designs. Learning (all in schools with low average science achievement scores) was 
greater when children were taught with four variables (63% correct) than when they were taught 
with only two variables (chance level performance of 53% correct). The opportunity to plan the 
design before constructing it had no effect on performance. 

Conclusions: 

Description of conclusions, recommendations, and limitations based on findings. 

The research described here clarifies several important components that influence 
learning of the Control of Variables Strategy by 4 th - grade students. First, combining the 
components of teacher-led instruction and student-led manipulation benefits learning in a way 
that neither element alone achieves. Second, opportunities for student-teacher interaction 
produced more learning than instruction that did not call for student response. Third, attempts to 
increase learning among children from schools low in science achievement by “simplifying” 

CVS instruction were not successful. That is, focusing instruction on unconfounded experiments 
(“positive” examples in Study 3) or reducing the number of variables that students needed to 
attend to during instruction (Study 4) were not as effective as instruction that began with 
confounded experiments or required children to attend to more variables, particularly among 
children from schools lower in average science achievement. Both of the latter conditions may 
place more stress on the rationale for CVS and on why all variables other than the focal one must 
be controlled. Fourth, the benefits of CVS instruction are best consolidated and maintained over 
time when instruction precedes the student-led design of ramps experiments and CVS testing. 
Finally, the major challenge for future efforts to improve CVS understanding is to improve the 
effectiveness of instruction for children who may be less prepared for science instruction. For 
these children, it may be essential to place more emphasis on promoting understanding of the 
goal of scientific experiments prior to instruction in the manipulation and control of variables. 
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